MILAB-6303: tree-sync introspection metrics + stop-marker follow-up fix - #1760
Conversation
Extend the MI_LOG_TREE_STAT counters with per-cycle re-fetch classification computed in updateFromResourceData: new, changed and unchanged resources, wasted downlink bytes, BFS fetches spent on unchanged resources, and changes that re-streamed stable metadata. Quantifies how much of each poll re-fetches resources the client already holds. Reuses the existing changed flag, no extra work per resource.
Expand MI_LOG_TREE_STAT into full per-path introspection: streaming detail (rounds, resource/stop-marker frames, follow-up seeds, traverse-stopped), BFS detail (resources requested, not found), and a per-cycle change breakdown (fields added/removed/changed, kv, ready flips, locks, errors, duplicates resolved, resources marked final), alongside the existing cross-cycle duplication counters. Fix the resourceTree stop-marker follow-up to propagate traverseStopRules so the retry stops at the same boundaries instead of traversing the stop-marked subtrees unbounded. Assert that one follow-up round resolves every stop marker (throws if a deeper stop-marker chain remains, which would mean the retry must become a loop).
🦋 Changeset detectedLatest commit: 8b226cf The changes in this PR will be included in the next version bump. This PR includes changesets to release 5 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
There was a problem hiding this comment.
Code Review
This pull request instruments tree-sync statistics (MI_LOG_TREE_STAT) to provide detailed introspection on cross-cycle duplication, per-path details, and cycle changes, while also fixing stop-marker follow-up propagation. The review comments correctly identify a critical issue where adding, modifying, or deleting KV entries does not set the changed flag to true. This omission causes incorrect state versioning, inaccurate introspection metrics, and prevents proper final state evaluation.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1760 +/- ##
==========================================
+ Coverage 53.49% 53.64% +0.15%
==========================================
Files 365 366 +1
Lines 19524 19679 +155
Branches 4291 4341 +50
==========================================
+ Hits 10444 10557 +113
- Misses 7792 7818 +26
- Partials 1288 1304 +16 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
KV mutations previously did not set the changed flag, so they never bumped dataVersion, skipped isFinalPredicate re-evaluation, and were miscounted as unchanged re-fetches in the tree-sync stats. Set changed on KV add, modify, and delete so versioning, finality, and the duplication metrics are correct. Addresses PR review.
Comment out the throw on unresolved stop-marker seeds after the follow-up round while the deeper-chain case is under investigation. Keep the traverseStopRules propagation and leave re-enable instructions inline.
…hains Propagating traverseStopRules to the follow-up bounded the retry but left stop-marker chains deeper than one level unresolved: a loaded resource referenced a deeper stop-marked resource that was never fetched, so updateFromResourceData threw "orphan resource" on open. Loop the follow-up (each round with the same stop rules, deduping fetched ids) until no stop-marker seeds remain, so everything referenced is loaded. Replaces the disabled one-shot assertion.
Reapplying traverseStopRules on the follow-up re-flagged the very resources being retried as stop markers (the rule is finality-based and they are server-final), so their state never loaded and resources referencing them threw "orphan resource" on open. Fetch follow-up seeds plainly; keep the loop only to resolve any residual stop markers. Update the changeset accordingly.
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
Per review: replace the positional boolean arg on updateFromResourceData with
an options object ({ allowOrphanInputs, stat }), updating the one caller; and
build formatTreeLoadingStat as a single template literal instead of concatenating.
Tree-sync instrumentation and a stop-marker follow-up fix, backing the MILAB-6303 investigation (Desktop App on slow / distant networks). Spec: milaboratory/text#190.
What this adds
Full introspection of the tree-sync load via
MI_LOG_TREE_STAT, computed by reusing the state machine's existingchangedflag (negligible per-resource cost):This measured that 52-76% of every polling cycle re-fetches unchanged resources (trees changed by only 2-9 resources over 3 min) on a heavy project.
Fix
The
resourceTreestop-marker follow-up now propagatestraverseStopRules, so the retry stops at the same boundaries instead of traversing the stop-marked subtrees unbounded. It also asserts that one follow-up round resolves every stop marker (throws if a deeper stop-marker chain remains, which would mean the retry must become a loop).Status
Draft. Type-check clean; pl-tree unit tests pass (the one skipped failure needs a live
PL_ADDRESS). No end-to-end validation of the stop-rule change under a deep tree yet; the assertion is the tripwire for that.Greptile Summary
Adds detailed tree-synchronization introspection and changes how streaming stop markers and KV mutations are reconciled.
ResourceUpdateStat/TreeLoadingStat— counters describing fetched resources, duplicate traffic, mutation categories, streaming frames, BFS requests, and stop-marker follow-ups; this PR defines and populates the expanded metrics.changed— the per-resource update flag controllingdataVersion, finality reevaluation, and changed-versus-unchanged statistics; this PR extends it to KV mutations but misses dynamic-field removal.metadataChanged— the classifier separating structural metadata churn from value/flag-only changes; this PR introduces it for metrics but does not mark field-type transitions.traverseStopRules— backend filters that stop traversal at configured resource boundaries; the initial stream applies them, while the new stop-marker follow-up loop deliberately omits them.stopMarker— a streaming frame identifying a resource whose traversal was stopped and whose state may require a follow-up fetch; this PR adds multi-round resolution and related counters.finalResources— locally immutable resources excluded from additional retrieval; this PR adds statistics around skipped final resources and uses the set while processing follow-up streams.Confidence Score: 2/5
The PR should not merge until stop-marker follow-ups preserve descendant traversal boundaries and field mutations are classified consistently.
Follow-up streaming can retrieve subtrees beyond the configured stop rules, while dynamic-field removals and field-type transitions produce incorrect state-version/finality handling or misleading diagnostics.
Files Needing Attention: lib/node/pl-tree/src/sync.ts, lib/node/pl-tree/src/state.ts
Important Files Changed
Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart TD A[Construct loading request] --> B[Initial resourceTree with stop rules] B --> C{Frame kind} C -->|Resource| D[Collect resource state and metrics] C -->|Stop marker already final| E[Skip] C -->|Unknown stop marker| F[Queue follow-up seed] F --> G[Follow-up resourceTree without stop rules] G --> C D --> H[updateFromResourceData] H --> I{Resource changed?} I -->|Yes| J[Advance dataVersion and reevaluate finality] I -->|No| K[Count unchanged bytes]Prompt To Fix All With AI
Reviews (1): Last reviewed commit: "MILAB-6303: fetch stop-marker follow-up ..." | Re-trigger Greptile
Context used: